Online Constrained Optimization over Time Varying Renewal Systems: An Empirical Method

This paper considers constrained optimization over a renewal system. A controller observes a random event at the beginning of each renewal frame and then chooses an action that affects the duration of the frame, the amount of resources used, and a penalty metric. The goal is to make frame-wise decisions so as to minimize the time average penalty subject to time average resource constraints. This problem has applications to task processing and communication in data networks, as well as to certain classes of Markov decision problems. We formulate the problem as a dynamic fractional program and propose an online algorithm which adopts an empirical accumulation as a feedback parameter. Prior work considers a ratio method that needs statistical knowledge of the random events. A key feature of the proposed algorithm is that it does not require knowledge of the statistics of the random events. We prove the algorithm satisfies the desired constraints and achieves O( ) near optimality with probability 1.